
<!DOCTYPE html>
<html><head lang="en"><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    
    <meta http-equiv="x-ua-compatible" content="ie=edge">

    <title>Dual Camera</title>

    <meta name="description" content="">
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <!-- <base href="/"> -->

<!--     <link rel="apple-touch-icon" href="apple-touch-icon.png"> -->
  <link rel="icon" type="image/png" href="img/lens-icon.png">
    <!-- Place favicon.ico in the root directory -->

    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.css">
    <link rel="stylesheet" href="css/app.css">

    <link rel="stylesheet" href="css/bootstrap.min.css">
    
    <!-- Global site tag (gtag.js) - Google Analytics -->
    <script async src="https://www.googletagmanager.com/gtag/js?id=UA-131406202-2"></script>
    <script>
    window.dataLayer = window.dataLayer || [];
    function gtag(){dataLayer.push(arguments);}
    gtag('js', new Date());

    gtag('config', 'UA-131406202-2');
    </script>


<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/1.5.3/clipboard.min.js"></script>
    
    <script src="js/app.js"></script>
</head>

<body>
    <div class="container" id="main">
        <div class="row">
            <h1 class="col-md-12 text-center">
                A Dual Camera System for High Spatiotemporal
                    <br>Resolution Video Acquisition<br> 
                <small>
                    <a href="https://vision.nju.edu.cn/">
                    Vision Lab, Nanjing University
                    </a>
                </small>
            </h1>
        </div>
        <div class="row">
            <div class="col-md-12 text-center">
                <ul class="list-inline">
                    <li>
                        <a href="https://chengming2.github.io">
                          Ming Cheng
                        </a>
                    </li>
                    <li>
                        <a href="https://vision.nju.edu.cn/fc/d3/c29470a457939/page.htm">
                          Zhan Ma
                        </a>
                    </li>
                    <li>
                        <a href="https://intra.ece.ucr.edu/~sasif/index.html">
                          M. Salman Asif
                        </a>
                    </li>
                    <li>
                        <a href="http://english.seiee.sjtu.edu.cn/english/detail/2737_1313.htm">
                          Yiling Xu
                        </a>
                    </li>
                    <body>
                          Haojie Liu
                    </body>
                    <li>
                        <a href="https://sites.google.com/view/wenbobao/home">
                          Wenbo Bao
                        </a>
                    </li>
                    <body>
                          Jun Sun
                    </body>
                </ul>
            </div>
        </div>

        <div class="row">
            <div class="col-md-8 col-md-offset-2">
                <!--<img src="images/pami.jpg" class="img-responsive" alt="overview"><br>
                <div align="center">
                    <img src="images/0.gif" height="222"/><img src="images/1.gif" height="222"/><br><img src="images/2.gif" height="160"/><img src="images/3.gif" height="160"/>  
                </div>-->
                <div align="center">
                    <img src="images/fi.gif" height="195"/>
                </div>
                <body align="center">
                    (a) Frame interpolation
                </body><br>
                <div align="center">
                    <img src="images/AWnet.gif" height="195"/>
                </div>
                <body align="center">
                    (b) AWnet
                </body>
                <h3>
                    Abstract
                </h3>
                <p class="text-justify">
                    This paper presents a dual camera system for high spatiotemporal resolution (HSTR) video acquisition, where one camera shoots a video with high spatial resolution and low frame rate (HSR-LFR) and another one captures a low spatial resolution and high frame rate (LSR-HFR) video. Our main goal is to combine videos from LSR-HFR and HSR-LFR cameras to create an HSTR video. We propose an end-to-end learning framework, AWnet, mainly consisting of a FlowNet and a FusionNet that learn an adaptive weighting function in pixel domain to combine inputs in a frame recurrent fashion. To improve the reconstruction quality for cameras used in reality, we also introduce noise regularization under the same framework. Our method has demonstrated noticeable performance gains in terms of both objective PSNR measurement in simulation with different publicly available video and light-field datasets and subjective evaluation with real data captured by dual iPhone 7 and Grasshopper3 cameras. Ablation studies are further conducted to investigate and explore various aspects (such as reference structure, camera parallax, exposure time, etc) of our system to fully understand its capability for potential applications.
                </p>
            </div>
        </div>
        
        <div class="row">
            <div class="col-md-8 col-md-offset-2 text-center">
                <h3>
                    
                </h3>
                <ul class="nav nav-pills nav-justified">
                    <li>
                        <a href="https://arxiv.org/abs/1909.13051">
                        <img src="images/paper.jpg" height="120px"><br>
                            <h4><strong>Paper</strong></h4>
                        </a>
                    </li>
                    <li>
                        <a href="http://yun.nju.edu.cn/f/2c34ad8492/?raw=1">
                        <img src="images/youtube_icon_dark.png" height="120px"><br>
                            <h4><strong>Video Demo</strong></h4>
                        </a>
                    </li>
                    <li>
                        <a href="https://github.com/NJUVISION/AWnet">
                        <img src="images/github_pad.png" height="120px"><br>
                            <h4><strong>Code</strong></h4>
                        </a>
                    </li>
                </ul>
            </div>
        </div>
<!--        <div class="row">
            <div class="col-md-8 col-md-offset-2">
                <h3>
                    <a href="http://yun.nju.edu.cn/f/2c34ad8492/?raw=1">
                    Video Demos
                    </a>
                </h3>
            </div>
        </div>
 
        <div class="row">
            <div class="col-md-8 col-md-offset-2">
                <h3>
                    Video Demos
                </h3>
                <body>
                    Different illumination conditions:
                </body>
                     <li>
                        <a href="http://yun.nju.edu.cn/f/5087da2041/?raw=1">
                          High Light Illumination
                        </a>
                    </li>
                    <li>
                        <a href="http://yun.nju.edu.cn/f/c251103e67/?raw=1">
                          Medium Light Illumination
                        </a>
                    </li>
                    <li>
                        <a href="http://yun.nju.edu.cn/f/42a121985b/?raw=1">
                          Low Light Illumination
                        </a>
                    </li>
                <br>
                <body>
                    Single-Reference vs Multi-Reference:
                </body>
                <br>
                    <li>
                        <a href="http://yun.nju.edu.cn/f/f8c3604d6a/?raw=1">
                          Simulated data
                        </a>
                    </li>
                    <li>
                        <a href="http://yun.nju.edu.cn/f/76f3cea6da/?raw=1">
                          Real data
                        </a>
                    </li>
 
            </div>
        </div>
 -->   
        <div class="row">
            <div class="col-md-8 col-md-offset-2">
                <h3 align="left">
                    BibTeX
                </h3>
                <div class="form-group col-md-10 col-md-offset-1" align="left">
                    <div class="CodeMirror cm-s-default CodeMirror-wrap">
                        <div style="overflow: hidden; position: relative; width: 3px; height: 0px; top: 4px; left: 4px;">
                            <textarea autocorrect="off" autocapitalize="off" spellcheck="false" style="position: absolute; padding: 0px; width: 1000px; height: 1em; outline: none;" tabindex="0">
                            </textarea>
                        </div>
                        <div class="CodeMirror-vscrollbar" cm-not-content="true"><div style="min-width: 1px; height: 0px;">
                        </div>
                    </div>
                    <div class="CodeMirror-hscrollbar" cm-not-content="true">
                        <div style="height: 100%; min-height: 1px; width: 0px;">
                        </div>
                     </div>
                        <div class="CodeMirror-scrollbar-filler" cm-not-content="true">
                        </div>
                        <div class="CodeMirror-gutter-filler" cm-not-content="true">
                        </div>
                        <div class="CodeMirror-scroll">
                            <div class="CodeMirror-sizer" style="margin-left: 0px; margin-bottom: -17px; border-right-width: 13px; min-height: 100px; padding-right: 0px; padding-bottom: 0px;">
                                <div style="position: relative; top: 0px;">
                                    <div class="CodeMirror-lines">
                                        <div style="position: relative; outline: none;">
                                            <div class="CodeMirror-measure"><span><span>​</span>x</span></div>
                                            <div class="CodeMirror-measure"></div>
                                            <div style="position: relative; z-index: 1;"></div>
                                            <div class="CodeMirror-cursors">
                                                <div class="CodeMirror-cursor" style="left: 4px; top: 0px; height: 16.6667px;">&nbsp;</div></div>
                                            <div class="CodeMirror-code" style=""><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;"><span cm-text="">​</span></span></pre><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;">@article{cheng2019dual,</span></pre><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;">    title={A Dual Camera System for High Spatiotemporal Resolution Video Acquisition},</span></pre><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;">    author={Cheng, Ming and Ma, Zhan and Asif, M Salman and Xu, Yiling and Liu, Haojie and Bao, Wenbo and Sun, Jun},</span></pre><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;">    journal={arXiv preprint arXiv:1909.13051},</span></pre><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;">    year={2019},</span></pre><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;">}</span></pre><pre class=" CodeMirror-line "><span style="padding-right: 0.1px;"><span cm-text="">​</span></span></pre></div></div></div></div></div><div style="position: absolute; height: 13px; width: 1px; top: 100px;"></div><div class="CodeMirror-gutters" style="display: none; height: 188px;"></div>
                        <body>
                            Correspondence: Zhan Ma ( mazhan@nju.edu.cn)
                        </body>
                        </div>
                    </div>
                </div>
            </div>
        </div>
    </div>
</body></html>
